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Gaussian Affine Feature Detector 



Xiaopeng Xu, Xiaochun Zhang 



Abstract — A new method is proposed to get image features' geometric information. Using Gaussian as an input signal, a 
theoretical optimal solution to calculate feature's affine shape is proposed. Based on analytic result of a feature model, the 
method is different from conventional iterative approaches. From the model, feature's parameters such as position, orientation, 
background luminance, contrast, area and aspect ratio can be extracted. 

Tested with synthesized and benchmark data, the method achieves or outperforms existing approaches in term of accuracy, 
speed and stability. The method can detect small, long or thin objects precisely, and works well under general conditions, such 
as for low contrast, blurred or noisy images. 



Index Terms — LoG, DoG, differential geometry, Hessian, Harris, affine, Fourier, Laplacian. 
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1 Introduction 

Detecting two dimension signals is more difficult than one 
dimension ones and many heuristic algorithms are proposed 
to deal with it. However, following appearance of scale 
space theory |[T2l . |7|, many effective feature detectors 
• come into being |l5l|. 
Q Originally, scale space theory is proposed by physi- 
1— 'cists, and developed by computer scientists. It is studied 
^ thoroughly from view point of vision and mathematics, 
^ and a consistent way to find new detector had been built 
121. Many successful feature detectors are built upon scale 
space, including |3|, |15|. 

An additional dimension is introduced in scale space, 
namely scale dimension. In order to get image's informa- 
tion, such as affine shape parameters, some methods |9|, 
O, m iteratively search in scale dimension. They are based 
^ on fix point theory: they will finally get a solution if there 
• • is one. In practice, however, these methods have several 
^ ^ drawbacks, including, 

• waste lots of candidate features; 
very slow; 

get abundant duplicated or false features. 
To overcome these drawbacks, we propose a new method 
based on analytic solution. It achieves or outperforms 
iterative methods with much less computation resources. 

Some feature detectors ifTOli , Isl are ideal for noise free 
images, but are incapable of blurred or noisy images. The 
proposed detector is more robust with similar performance. 

Features' position, orientation, background luminance, 
contrast, area and aspect ratio can also be extracted from 
images. Until recently, information such as background 
luminance contrast are not commonly used in feature ex- 
traction. Others including area, orientation and aspect ratio 
are studied extensively, but with a limited accuracy. 

In this paper, a feature model is proposed, and the above 
mentioned parameters will be calculated. 
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2 Gaussian Affine Shape 

In this section, firstly a feature model is proposed, and then 
analytic result is derived based on the model to get various 
parameters. 



2.1 Feature Model 

From a view point of systematology, images, feature extrac- 
tor and features correspond to input, system, and output. We 
need build a system that can transform input to output. In 
another words, image is system input and feature parame- 
ters are output. In order to study behavior of the system, we 
need define input signals. As it is not possible to build an 
all-purpose feature extractor, we will concentrate on some 
specific image signals. Since (two dimensional) Gaussian 
has nice analytic properties and simple form, it is chosen as 
input signal. As to be shown later, Gaussian based model 
will filter out high frequency signal, hence ideal for noisy 
images. 

Based on above mentioned idea, image surface is mod- 
eled as Gaussian function, as shown in Fig. [T] In this 
way, image feature parameters are related to Gaussian 
parameters, including orientation, long and short radii, 
baseline height and contrast. Traditionally, baseline height 
and contrast are not considered in feature extraction, they 
are included for completeness. The signal can be defined as 
Equ. [T] Parameters and model variables are listed in Tab. [T] 



(1) 



Before continuing, it is helpful to clarify a fact, that 
is, rotating an image will not affect our discussion. This 
fact greatly simplifies our deduction. It is proved in Ap- 
pendix. [a| 

Known the fact, two-dimensional axis-aligned Gaussian 
will be used as input signal, as shown in Equ. |2] For this 
function, we need get value of a, (3, c and d. 



l{x,y;a,(3,c,d) = c e 



Mi 



(2) 
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Fig. 1 . Model of input signal, whose baseline height is 
and contrast, is 1. 

TABLE 1 
Factors of model 



Factors 


Parameters 


Contrast 


c 


Baseline height 


d 


Long radius 




Short radius 


a 


Nominal radius 




Aspect ratio 


OL 


Orientation 


e 


LoG detected scale 


a 



2.2 Feature Detection 

Before computing parameters, we need detect feature's 
position. There exists many feature detectors, we need 
choose the one that has good performance and soHd mathe- 
matical foundation. It will be chosen from rotation invariant 
differential operator family. As defined in Equ. [3j LoG 
detector is a good candidate, it is very stable, has fast 
implementation, and is Gaussian based. The last one is 
the most important reason, because input signal is also a 
Gaussian, and they may have close relation. 



LoG = V • VG, G 



27rcr2 



dx (/l * /2) = /l * dxj2 
dxfl * /2 



(3) 



(4) 



Using Equ. [5] which is based on Equ. |4] we can define 
LoG operation on image function, as shown in Equ. [6] 



dx,x{fl * /2) = dx^xfl * /2 = /l * dx,xf2 
dx,y{fl * /2) = dx^yfl * /2 = /l * dx,yf2 

dy,y{fl * /2) = dy^yfl * /2 = /l * dy^yf2 



(5) 



LoG * / = {dx,xG + dy^yG) * / 

= dx,xG ^ I ^ dy^yG ^ I 

V- V(G*/) 



As shown in Equ. [7] convolving I with G is another 
Gaussian, which is called (Gaussian) scale space. For 
zero shifted /, its Laplacian will get extreme at origin. 
Normalizing this value will get normalized Laplacian of 
Gaussian operation upon I, which is basis of some feature 
extractors. 



cap 



(7) 



Applying LoG to image to get extreme points, and with 
information provided by G * /, we need get radii (standard 
deviations) of original input (Gaussian) function /. 

2.3 Parameters Calculation 

As shown before, image / can be considered as a surface 
in three-dimensional space. From differential geometry, we 
know its hessian matrix directly relates to principal curva- 
tures and principal directions, and for Gaussian function, 
principal curvatures connect with its standard deviations. 
In one word, eigenvalues of hessian matrix relate to radii 
and eigenvectors relate to directions. We also know that two 
principal directions are perpendicular to one another. Based 
on these facts, we will derive formulas for parameters. 

Obviously, convolving / with isotropic Gaussian will not 
change principal direction. For extreme point, we can use 
G*/'s principal direction as /'s principal direction. For the 
case of axis-aligned Gaussian, we already know principal 
directions, otherwise, compute eigenvectors. 

Remaining question is, giving information of G * / , how 
to get /'s radii a and (3, its contrast c and baseline height 
d. 

Here, we will exploit a fact, that LoG can detect Gaussian 
at one and only one scale. In another word, every a and (3 
pair must produce one and only one a, as shown in Equ. [8] 
If analytic form of / is determined, we can recover a and 
/3 from a. 



(8) 



For any input image, £ is fixed. LoG will detect extreme 
point in a fixed scale a. Let us denote k = and h = -. 

Apply normalized LoG operator to I, and substitute P = 
ak and a = ah, and let x = 0, y = 0, we get Equ. |9] 



cr2V-V(G*/) 



x=0,y=0 



ch^k{2^h^ {l^P)) 



(9) 



((l + /l2)(l^/,2p))3/2 

Let c be Constance 1 and draw this expression in Fig. [2] 
It is clearly shown that for A: > 1, extreme of LoG * / is 
located on a smooth ridge. 

For a fixed k, at extreme point, the formula's one order 
derivative will be zero. After some calculation, we can get 



Equ. 10 



(6) 



-4-2h^ (1 + e)-h^ (1 - 6^ + i^4)+2/i^ + k^ 



(10) 
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ratio of eigenvalue 



Fig. 2. k and h constrained 



Fig. 3. Relation of H and ratio of eigenvalues 



To solve this equation, let k'^ = K and h'^ = H, and we 



get two order equation Equ. 1 1 
It is easy to solve, as Equ. [12] 



(11) 



H{-1^2H) 



(12) 



i^(-l + 2i^) 

Known constraint of K and H, we need more informa- 
tion to get their values. As mentioned above, eigenvalues of 
hessian matrix relate to radii closely. We calculate hessian 



matrix over scale space, as shown in Equ. 13 

(G*/) 



d 
3 



d:r. 



y,y 



(13) 



As before, we calculate eigenvalues of this matrix, and 
let X = 0, 7/ = 0. Since our discuss based Gaussian, we can 



get analytic solution of two eigenvalues, shown in Equ. 14 



ei 



e2 



ch^k 



(1 + /l2)3/2 ^/Y^J^^2 
Ch^k 



(14) 



VTT7?(l + /i2/.2)3/2^2 

These two eigenvalues have complicated form, but their 
ratio is simpler, shown in Equ. [15] 



ei 
e2 



(15) 



From Equ. 15 we can solve for K, shown in Equ. 16 

Hr 



K 



-1 + r - 



H 



result is Equ. 17 



Combined Equ. 12 and Equ. 16 we can solve for H 



H 



2r(l + r) 

We draw this relation in Fig. [3] which shows detecting 
scale tends to be constancy as shape gets elongate. Simply 
put, elongating a shape contributes little to its detecting 
scale. 



Got H, k, h, a and (3 will be solved directly. 

c and d can also be solved in analytic form. Equ. [9] is 
used to get c. Because d is constant component of scale 
space, it will disappear by differential operation; therefore 
can only be solved in scale space itself. Let x = and 



= in G * /, we will get Equ. [18] so d can be solved 
upon extreme point of scale space. 



G*4= 



--0,y=0 



1 + i 



1 



1 

HK 



(18) 



Until now, we have calculated all parameters of the 
zero shifted and axis aligned Gaussian. Because axis can 
be shifted or rotated, our discussion will be applied to 
Gaussian of any position or rotation. We will summary the 
steps of our algorithm. 

• Detect extreme point in normalized LoG space, and 
get its a. 

• compute hessian matrix of extreme point in corre- 
sponding scale space 

• compute eigenvectors as principal directions of the 
point. 

• compute eigenvalues, let absolute larger one divide 
smaller one, and represented as r 



• use Equ. [T7] to compute H, Equ. [16] to compute K, 
and use a = \/Hcf^P = \fKoL to compute other 
parameters, use Equ. [9] and Equ. 18 to solve for 
contrast and baseline height. 

2.4 Data Transformation 

Until now, signal's radii and angle are extracted. In order to 
comparing with other methods' results, we depend on some 
publicly available tools. Therefore radii and angle need 
to be transformed to a common form, such as symmetric 



Q^^j positive definite matrix, as shown in Equ. [19 



X y 
y z 



(19) 



If let be signal's orientation, and t = arctan(6>), in a 



(17) 

similar way as before, we get Equ. \M 



l+t2 

l+t2 



(20) 
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TABLE 2 
Test Image condition 



Parameter 


Range 


c 


[-255,255] 


d 


[0, 255] 




[5,40] 


OL 


[1,30] 


e 


[~ 2"' 2"] 



Fig. 4. relation of k and r 



3 Implementation Details 

In this section, some important implementation details are 
outlined. 



Ours 



Hessian-Affine 



3.1 Approximation and Adjustment 

As shown in Equ. [21] LoG can be implemented by DoG, 
and together with pyramid algorithm, which makes pro- 
posed method ready for application. We use similar DoG 
pyramid as Lowe's. Extremum of DoG should be adjusted 
by a constant multiplier, for its value is used to compute c 
and d. 

G{x, y, ka) - G(x, y, a) ^ {k - l)cr^ V^G (21) 



3.2 Removal of False Features 

Tested with synthesized data, we found one common prob- 
lem among several (affine) feature detectors, that is, for 
a single Gaussian signal, often there are several features 
detected out. Some of them have similar radii and orienta- 
tions, located around true position, as shown in Fig. [5] and 
Fig. [9] Others are false features arisen from noise, as shown 
in Fig. [9] and Fig. \T0 



Harris-Affine 



Mser 



In practice, we found a large part of false features coming 
from sampling and digitization process, that is to say, they 
are small sized, low contrast features. True features seldom 
have such properties. Therefore features with small value 
of c, a and /3 are considered as noises. 

3.3 Detector Threshold 

Like SIFT, we uses ratio of principal curvatures (ratio of 
hessian's eigenvalues, or r in our method) to remove points 
on valley or ridge. To accept more features, the ratio needs 



to be refined. Combining Equ. [T7] and Equ. [16) with K 
A:^, we have Equ. 



22 



k = 



3r3 



(22) 



3 + r2 

We have drawn relation of k and r in Fig. [4] For aspect 
ratio k to be as high as 40, r need at least to be 535 
theoretically. The r in Equ. [23] is threshold of features. 

TT{Hf (61 + 62)2 (r62 + 62) (r + 1)^ 



Det(i^) 



6162 



r62^ 



(23) 



Fig. 5. Typical results for an isotropic Gaussian 

4 Experiments 

In order to evaluate performance of our method, we firstly 
test it with synthesized data. In this way, we will know true 
parameters and therefor can compare them with calculated 
ones. We will compare results of our method and others, 
including Harris-Affine, Hessian-Affine and Mser. Only 
common parameters such as orientation, long and short 
scale can be compared, because contrast and base height 
are unique provided by our method. Nevertheless, we will 
show the results alone. 

Gaussian will be used as test image. Image size is 
256x256, and gray scale level is 256. Our method can detect 
a large range of parameters, and Tab.[2]lists parameters used 
in experiments. 

4.1 Results of ideal signals 

As demostrated in Fig. [5j Hessian-Affine and Harris-Affine 
tend to detect duplicated features. Fig. [6] and Fig. [7] show, 
for noise free Gaussian signal, Mser has highest accu- 
racy for detecting position and aspect ratio. Our method 
achieves similar results as MSer. Compared with Harris- 
Affine, Hessian-Affine gets better results. Both Mser and 
our method can detect signals of high aspect ratio, but 
Hessian-Affine and Harris-Affine are limited to low aspect 
ratio signals. 

Our method is to compute original parameters from 
blurred output image. For very long and thin shapes, our 
method may slightly underestimate true aspect ratio, as 
shown in Fig [7] 

As shown in Fig.[8j our method and Mser achieve highest 
accuracy for detecting short radii. However, in addition to 
true signals, Mser often finds small concentric signals. 
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MSer 
Ours 

Harris-Affine 
Hessian-Affine 



it* + 



true aspect ratio 



j iyi|tn:l!!=;tttftt 



MSer 
Ours 

Harris-Affine 
Hessian-Affine 



• t * 



true aspect ratio 



Fig. 6. Position inaccuracy 



Fig. 8. Short radius accuracy 



£ 1-3 
i- 1-2 



MSer 
Ours 

Harris-Affine 
Hessian-Affine 



I 



real aspect ratio 



Fig. 7. Aspect ratio accuracy 



In conclusion, for ideal noise free Gaussian, Mser get 
best results, and ours is similar to that of Mser. Hessian- 
Affine and Harris-Affine are not as stable as Mser and ours. 

4.2 Results of noisy signals 

Fig. |9] is a typical noisy image, and Mser is the most 
sensitive to noise. Even a small amount of noise can impact 



Mser seriously. Fig. [T0| is distance of true and detected 
points. It is difficult for Mser to differentiate noises from 
true signals. Therefore we only compare Hessian-Affine, 
Harris-Affine and ours for noisy images. 

Fig. [11] Fig. 



As shown in Fig. 10 



11 our method 



performs well when other methods reach their limits. 
Using Mikolajczyk's evaluation images and toolbox, we 



get repeatability in Fig. [13] For these noisy free images, 
Mser get highest accuracy, and Hessian-Affine, Harris- 
Affine and our methods have similar results. Our 1 and 
2 are results of different thresholds. 

Fig. [14] is detecting results of graffiti under different view 
angle. Compared with Hessian-Affine and Harris-Affine, 
Mser and ours detect fewer features. It seems that the 
former two detect many redundant features. Compared with 
ours, Mser tends to detect many small features. 

5 Conclusion 

In this paper, we have proposed a new feature detector. 
Compared with other methods, it is very stable, accurate 



Ours 



Hessian-Affine 



iP 



IVIser 



Fig. 9. Typical results for noisy Gaussian 



and quick. Tested with Gaussian, for ideal noisy free signal, 
our method produces one of the best results, and for noisy 
signal, it outperforms others significantly. The proposed 
method can also extracts parameters unavailable for other 
methods, such as contrast and baseline height. 

Test with benchmark images, the method get similar 
repeatability as Harris-Affine and Hessian-Affine. 



Appendix A 

Proof of Rotation Invariant for Image 
Surface 

Let F be Fourier operator, and / be an input function; 
Fourier transform is shown in Equ. [24] 

F o f{x) = / /(x)e-2-<-.«>dx = /(O (24) 



If input function rotates in x space, and let ^ = Rx, its 
Fourier transform will also rotate same angle, as shown in 
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I i #i » t » t : 



MSer 
Ours 

Harris-Affine 
Hessian-Affine 



4> # ^ = 



true aspect ratio 



■ + + t ; 



Ours 

Harris-Affine 
Hessian-Affine 



true aspect ratio 



Fig. 10. Position inaccuracy for noisy Gaussian 



Fig. 12. Short radius accuracy for noisy Gaussian 



true aspect ratio 



+ Ours 

+ Harris-Affine 

O Hessian-Affine 



10 



12 



Fig. 1 1 . Aspect ratio accuracy for noisy Gaussian 




30 40 50 

viewpoint angle 



Equ. 25 



Fof{Rx)= /(i?a;)e-2'^*<^'«)da; 

J OO 
POO 

J OO 

poo rj. 

J OO 



-L 



POO 

J OO 
POO 



f{y)e 
f{y)e 



-27riy 



dy 



= fm (25) 
Convolution in space domain can be implemented by 



multiplication in ^ domain, as shown in Equ. 26 



fix) * g{x) ^ fiOgiO = m ^ K^) (26) 



f{Rx) * g{Rx) ^ f{R09{R0 = K^O ^ H^^) (27) 

Using Equ. |24j Equ. [25] and Equ. |26j we can get Equ. [27] 
It means if input and system are rotated with same angle, 
the output will also rotate the same angle. In one word. 



Fig. 13. Repeatebility comparasion. Our 1 and 2 have 
different threshold. 



output's geometrical property will not change on rotating 
input and system. 
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